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Executive Sumniary 



Purpose 



The Department of Defense plans to spend between $69 and S87 billion 
to develop and deploy the first of a multiple phase Strategic Defense 
System designed to shield the United States from a massive Soviet 
nuclear ballistic missile attack. At the request of the Chairman, Legisla- 
tion and National Security Subcommittee, House Committee on Govern- 
ment Operations, gao reviewed the Strategic Defense Initiative 
Organization's (sdio's) test and evaluation plans. These plans are to sup- 
port a 1993 executive decision by the President on deploying Phase I 
and a 1994-1995 acquisition decision by the Department of Defense to 
enter full-scale development. Specifically, gao was asked to determine 



what test and evaluation requirements were established for making 
development and deployment decisions for the first phase, and 



whether SDiO's approach will meet those requirements and support a 
decision on deployment 



Background 



The Strategic Defense System is being designed to destroy thousands of 
Incoming ballistic missiles and warheads. This will require detecting the 
missiles and warheads, discriminating them from hundreds of thousands 
of decoys, precisely tracking the missiles and warheads, and then 
destroying them— all within about 35 minutes or less, and despite the 



System is, by far, the most complex, technologically challenging system 
ever attempted. As currently envisioned, a sophisticated communica- 
tions system will link together hundreds, or possibly thousands, of com- 
puter-operated components in space and on the ground- The system will 
consist of several subsystems (elements), which include surveillance 
satellites, space- and ground-based weapons, and ground-based subsys- 
tems to command and control the satellites and weapons. 

The Strategic Defense System is being developed in several phases. 
Phase I will be built using existing technologies; later phases would use 
more exotic technologies, such as lasers and particle beams. On Sep- 
tember 17, 1987, the Secretary of Defense directed that Phase I enter 
the concept demonstration and validation stage. This stage is important 
because enough information must be developed to show that the system 
is feasible before a decision is made to enter full-scale development. 
Because Phase I cannot be demonstrated outside of an actual ballistic 
missile attack, much of this information can be generated only through 
computer simulation and modeling. 
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In 1988, sdio formally adopted a design (architecture) for Phase I- How- 
ever, in January 1990 sdio changed the design dramatically by incorpo- 
rating a new space-based weapon, called Brilliant Pebbles. The idea 
behind Brilliant Pebbles is that thousands of small, relatively autono- 
mous weapons would be deployed to intercept Soviet intercontinental 
ballistic missiles. 



T?PSlllts in Rripf The President is scheduled to make a decision in 1993 on deploying 

Phase I. His options include deploying, delaying, or cancelling Phase I. 
Public Law 99-145 states that a Strategic Defense System may not be 
deployed in whole or in part unless the President certifies to the Con- 
gress that the system can fulfil! its mission, sdio may be able to justify a 
decision to delay or cancel Phase I; however, sdio will not be able to 
support an informed decision to deploy it To do so requires a stable 
design, sufficient testing and evaluation, and according to sdio, rninimum 
funding levels. However, it is highly unlikely any of these conditions 
will be met 

The addition of Brilliant Pebbles in January 1990 has reduced, changed, 
or eliminated the need for some of the space-based elements in the 1988 
design. At this point, it is uncertain exactly what Phase I will consist of 
in terms of elements or what functions they will perform. Additionally, 
while SDIO initially required real-time integrated system-level testing 
prior to a presidential deployment decision and a full-scale development 
decision, sdio officials now state that such detailed teste will not be con- 
ducted prior to either decision. Furthermore, according to sdio, because 
of fiscal year 1990 funding cuts, the full-scale development decision has 
been delayed until 1994-1995, and research efforts have been scaled 
back, further reducing the amount of information that will be available 
for a presidential deployment decision, sdio officials believe they will be 
able to support a presidential deployment decision with less information 
than was originally desired, but state that this will be done at increased 
risk. 

Phase I has not received the scrutiny and oversight that Defense ini- 
tially envisioned. The Defense Acquisition Board did not conduct its 
scheduled 1989 review, nor has it assessed how Brilliant Pebbles affects 
the design and test and evaluation requirements. The lack of effective 
agency oversight has contributed to the failure of other automated 
weapons systems, none of which matches the scale and technological 
complexity of Phase I. Consequently, any executive decision in 1993 to 
deploy Phase I would be premature and fraught with high risk. 
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Principal Findings 



SDI Design and Test Plans Several problems are hampering edio's design and test plans. First, Bril- 
in State of Flux ^ ant ^ >e ^ es radically changed the design and sent the program into a 

state of flux, sdio has not yet solidified the role of Brilliant Pebbles or 
what elements will be in the final design. This causes problems because 
without a stable design detailed Strategic Defense System requirements 
cannot be determined. Setting requirements is especially important for 
Phase I where weapon, sensor, and communication systems are interde- 
pendent Changing a requirement for a weapon system could signifi- 
cantly affect sensor and communicarions systems as well. Further, an 
unstable design increases the probability that system requirements will 
not be adequately determined and sufficient testing will not be per- 
formed to ensure that the system works. Nevertheless, Defense has 
requested S265 million for full-scale development of one of the sensor 
elements, the Boost Surveillance and Tracking System, for fiscal year 
1991 even though Brilliant Pebbles may eliminate or reduce the require- 
ments for this system. 

Second, sdio does not plan to conduct integrated system-level testing by 
the scheduled 1993 presidential decision. SMO-funded studies and test 
plans have cited the importance of running real-time integrated system- 
level tests, which combine computer simulation and actual hardware 
and software prototypes with human intervention. Furthermore, most 
of the system tests and evaluations so far have been based on the pre™ 
Brilliant Pebbles design. Consequently, much of the testing and analyses 
may no longer be relevant and will have to be repeated. 

Also, according to sdio, its ability to support an informed deployment 
decision depended on a minimum funding level of S4.6 billion for fiscal 
year 1990. However, funding was cut by 20 percent. According to sdio, 
such a cut will reduce the amount of research performed, delay sdio's 
full-scale development decision by 2 years, and reduce the amount of 
information available to the President Nevertheless, sdio officials have 
stated that they will be able to support a presidential decision on the 
program even though less information will be available. 
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Original Program Because of the complexity, uniqueness, and cost of Phase I, the Defense 

Oversight Reduced Acquisition Board initially was to follow a highly structured oversight 

to system of annual reviews looking at cost growth, changing require- 

ments, and test and evaluation results. However, the October 1989 pro- 
gram review scheduled by the Defense Acquisition Board did not occur. 
According to Defense officials, the review was cancelled because the 
fiscal year 1990 budget had not yet been finalized and because Defense 
was reassessing the amount of oversight needed for the Strategic 
Defense System- Even though a Defense Acquisition Board review is 
scheduled for June 15, 1990, major changes to the Phase I design and 
significant reductions to test and evaluation requirements have already 
occurred without the Board's review and evaluation. 



Commendations to 
the Secretary of 
De f ense 



gao recommends that the Secretary of Defense advise the President to 
defer a decision to deploy any element of the Phase I system until sdio 
has stabilized the design and has demonstrated the effectiveness of the 
system through integrated system-level tests in real time, using system 
hardware and software prototypes with human intervention. Further, 
the Secretary should ensure that required oversight by the Defense 
Acquisition Board be followed. The Board should more closely monitor 
system design, development, testing, and evaluation, gao is making 
other recommendations in chapter 4. 



Recommendation to 
the Congress 



<H0 recommends that the Congress not fund full-scale development for 
any element of Phase I, including S265 million for the Boost Surveillance 
and Tracking System in fiscal year 1991, until sdio has stabilized the 
design and has demonstrated the effectiveness of the system through 
integrated system-level tests in real time, using system hardware and 
software prototypes with human intervention. 



Agency Comments 



As requested by the Chairman's office, gao did not obtain official 
agency comments on a draft of this report However, gao discussed the 
information contained in this report with sdio officials and has incorpo- 
rated their comments where appropriate. 
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Introduction 



The Strategic defense Initiative (sdi) is a program to develop and deploy 
a Strategic Defense System to shield the United States against a Soviet 
nuclear ballistic missile attack. Research and development for the sdi 
program began in 1984, and in 1987 a Strategic Defense System was 
approved for acquisition- This system is to bedeveloped in multiple 
phases. Phase I, the subject of this report, is in the concept demonstra- 
tion and validation stage of the Department of Defense's acquisition 



The Strategic Defense System will be an expensive undertaking. The 
Strategic Defense Initiative Organization (sdio) cost estimates range 
from S69.6 to S87 billion for completion of Phase I development and 
deployment — with an additional 510.1 billion for Phase I operation and 
support until deployment is completed. While the cost of follow-on 
phases is unknown, sdio estimates that, in addition to the Phase I costs, 
approximately $26.4 billion will be needed for fiscal years 1990 through 
1994 to pursue non-Phase I research and development. GAO h?s recently 
reported to the Congress that the above figures are, at best, optimistic. 2 



fk>np<;ic;/Fvnlntif»n nf On March 23, 1983, President Reagan called for a comprehensive scien- 
%T cm '_r tific research effort to develop a system that would render nuclear hal- 

tu6 SDI PrOgT&ni Hstic missiles impotent and obsolete. In January 1984, sdi was 

established as a research and technology development program, and in 
April 1934, Defense formally chartered the Strategic Defense Initiative 
Organization as the agency responsible for managing Defense's efforts. 

After several years of research, sdio decided in 1987 to develop and 
deploy the system in phases, sdio felt that working in phases would 
allow SDIO to be prepared for an early deployment, if such a decision 
were made, and to respond to changing threats. Phase I of the system is 
to be based on currently available technologies, while later phases are to 
incorporate technologies that are expected to be available in the future, 
such as neutral particle beams and lasers. Phase I is net intended to be a 
"total defense", but is being designed to destroy a certain percentage of 
some intercontinental and sea-launched ballistic missiles (the exact per- 
centages are classified). Creating a full strategic defense capability will 



approach to designing, developing, implementing, and maintaining major weapons systems. (The tv 



'The Department of Defense's major system acquisition process is supposed to provii 
approach to designing, developing, implementing, and maintain! 
stages of the acquisition process arc discussed in appendix 1.) 

2 Strategie Defense Initiative Funding Needs Through Completi 
NSIAtfttMBK, Jan."E 1990)- 
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require the deployment of the follow-on phases. Therefore, a critical 
aspect of the first phase is the capability to evolve and support future 
phases. Further, the system is not being designed to destroy sea-artd air- 
launched cruise missiles. 3 Other costly systems independent of the SDi 
program will be needed to defend against these weapons. 

From the beginning, sdio has had one overall goal — to conduct a vig- 
orous research and technology program that would provide the basis for 
an informed decision regarding the deployment of a Strategic Defense 
System. The current Phase I program is intended to support an execu- 
tive decision on deployment by the President in 1993 and an acquisition 
decision on full-scale development by Defense in 1994 or 1995. 
According to SDIO test and evaluation officials, the President will have a 
range of options including deploying, delaying, or cancelling Phase I. 
However, if the President decides to deploy the system, Phase I develop- 
ment will not he consistent with Defense's prudent acquisition policies 
specifying that deployment decisions be made after Ml-scale 



Obiprfri Vf 0^ a The Strategic Defense System is based on a layered defense concept; the 

«* , A system is supposed to intercept a missile or a deployed warhead, as it 

Strategic Defense travels toward its target. The basic concept is that separate sensor, 

Svstem weapon, and command and control systems (sdio refers to these sepa- 

rate systems as elements) would be in f jace and on the ground. The 
system would be tied together by a complex communications network 
and a sophisticated computer-based command and control element 
During an attack, the system would have to function in an extremely 
hostile environment, including nuclear explosions and a concerted effort 
by the enemy to destroy the Strategic Defense System. 

The threat the system would be facing, along with the environment in 
which it would be operating, creates a unique and demanding challenge. 
First, the space-based elements of the Strategic Defense System would 
have to be able to detect and begin tracking thousands of missiles almost 
immediately after launch, and then intercept and destroy some of them. 
Those missiles that are not destroyed would release warheads, along 
with decoys intended to confuse the system, forcing the system to track 
hundreds of thousands of objects. Also at this point, space-based ele- 
ments of the system would have to distinguish the warheads from the 

3 Cnuse missiles sre guided missiles that have terrain-seeking radar arid fly at moderate speed in low 
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decoys, and intercept and destroy some of the warheads. The system 
would need to continue discriminating and tracking the surviving war- 
heads so that the ground-based elements could intercept more warheads 
before they hit the United Stetes. These functions would have to be 
tightly coordinated and performed in less than 35 minutes— all this with 
nuclear warheads exploding, anti-satellite weapons attacking the 
system, and the enemy trying to disrupt communications and computer 
operations. Figure 1.1 describes the phases of a ballistic missile attack. 



Figure 1.1: Phases of a Ballistic Missile A 




« ol Technology Assessment 
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Ppsrrmtinn nf Pha^P T In 1988 ' a PJiase l deagi, raade U P of seven elements, was approved by 
L-wuipwvii w± i auwc a the Defense Acquisition Board. The design consisted of two space-based 
sensors (Boost Surveillance and Tracking System and Space Surveillance 
and Tracking System); a space-based weapon (Space-Based Interceptor); 
two ground-based sensors (Ground Surveillance and Tracking System 
and Ground-Based Radar); and a ground-based weapon (Ground-Based 
Interceptor), all managed by a command and control system (Command 
Center Element). (Appendix H describes each element) The six surveil- 
lance and weapon elements would be highly interdependent and rely 
heavily on the command and control element, along with a complex com- 
munications system. The individual elements would work together as an 
integrated system to detect, track, discriminate, and destroy ballistic 
missiles and their warheads. At the heart of the system is a large, dis- 
tributed, real-time computer software system which, by some estimates, 
could have 40-100 million lines of code. The complexity and risk of this 
undertaking cannot be overstated- We have reported repeatedly on 
Defense systems that were far less complex, contained far fewer lines of 
code, but have yet to perform as intended.* 

In January 1990, sdio decided to include a new weapon concept, called 
Brilliant Pebbles, in Phase I. Brilliant Pebbles involves several thousand 
individual interceptors orbiting the earth in order to detect and destroy 
a target by smashing into it at high speed. Brilliant Pebbles is supposed 
to improve system survivability and reduce costs by dispersing 
thousands of space-based interceptors that are smaller and more auton- 
omous than the Space-Based Interceptor. The Space-Based Interceptor 
would house a number of interceptors, but unlike Brilliant Pebbl*s, it 
would rely on other satellites for tracking, targeting, and communica- 
tions. By making interceptors autonomous, they would no longer need to 
rely on other satellites to perform these functions. Further, because each 
interceptor would work independently, the system's survivability would 
be increased, in principle, because the loss of any one interceptor would 
not greatly affect the system's overall effectiveness. Figure 1 .2 shows 
Phase I with the six elements and Brilliant Pebbles. 
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Figure 1.2: Phase I Architecture Incorporating B 




BSTS is Boost Surveillance and Tracking System 
GBI is Ground-BaMd.lnterceptor 

GBRisGrountt-Basal'Radar' 1 - 

GSTS is Ground Surveillance and Tracking System 
SSTS is Space Surveillance Tracking System 
RVs are Reentry Vehicles (nuclear warheads) 



sdio funded three Brilliant Pebbles research and development studies 
during 1989 aridfl.990. Studies completed by the Defense Science Board 
and the JASONsr-a group of scientists who periodically provide tech- 
nical support to the Department of Defense—determined that the Bril- 
liant Pebbles concept was technically feasible. Along with these two 
studies, Defenseissued a Space-Based Architecture Study which 
reviewed the,fdur,space-based elements of Phase I — Space Surveillance 
and TrackmgiSi^tHniiBoost.Surveillance and Tracking System, the 
Space-Based^ln|ercept»ri|arid Brilliant Pebbles— and defined and justi- 
fied a recommeifded space-based architecture. The Space-Based Archi- 
tecture Study're^bmmended that research continue on both Brilliant 
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Pebbles and the Space-Based Interceptor and that modified versions of 
the Boost Surveillance and Tracking System and the Space Surveillance 
and Tracking System sensors be included. 



SDIO has decided to eliminate the Space-Based Interceptor from Phase I, 
daiming that Brilliant Pebbles will cut costs by 20 percent, that is, from 
$87 to $69.6 billion. However, it is unclear whether the costs for all 
Phase I requirements have been incorporated into the $69.6 billion. Fur- 
ther, neither figure includes the Costs for operating and maintaining the 
system. 



TrrmnrtanrA nf Crmrpnt ^ n September 17, 1987, the Secretary of Defense directed that Phase I 

impuiuuLLcyi \Amvcyb f the Strategic Defense System enter the concept demonstration and 

Demonstration ana validation stage of Defense's major systems acquisition process, sffiO's 

V&IidatiOIl aild ^asic acquisition strategy is to complete Phase I while researching later 

TWmieo A /vinioitirm P hases ^Strategic Defense System. After successfully completing con- 

ueieil&e AtqUiblUOIl ^ demonstration m $ validation, Phase I could enter full-scale 

Oversight 



According to the Defense Systems Management College, concept demon- 
stration and validation may be the most critical of all acquisition stages. 
The primary purpose of this stage is to validate the feasibility of Stra- 
tegic Defense System concepts and technologies, and show that the risk 
is low enough to enter full-scale development According to the College, 
the Secretary of Defense's decision to place a major system in full-scale 
development is extremely important During full-scale development, the 
system is built, tested, and ready for full-rate production. Not only will 
development con? ume enormous resources, but major systems, at this 
point, take on a li ; of their own and are seldom cancelled. 

To help Defense gain sufficient design and development information, 
major system acquisitions are reviewed by the Defense Acquisition 
Board (dab). The purpose of the dab, which is chaired by the Under Sec- 
retary of Defense for Acquisition, is to oversee major Defense acquisi- 
tions. The DAB reviews each acquisition stage to ensure that every 
program is ready to proceed into more advanced stages of development 
or production. Each stage of the acquisition process, as well as each 
milestone decision, is to be supported by test and evaluation. The pur- 
pose of test and evaluation is to help ensure the timely development, 
production, and fielding of systems that meet users' requirements and 
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perform as intended. The dab relies on information from test and evalu- 
ation to determine whether a system is ready to enter full-scale develop- 
ment. The MB reviews also ensure that plans for later stages follow 
sound acquisition management practices. 

The complexity, cost, and uniqueness of the Strategic Defense System 
has prompted the need for a more detailed program review process. For 
example, the dab developed a guidance document called the Imple- 
mentor, which provides a framework for additional oversight. Although 
formal dab reviews are usually only required at major milestones, the 
Implementor recommends yearly dab reviews. Further, sdio must submit 
additional documentation to the a\B to ensure program goals are being 



Constraints Affecting 
SDIO's Approach to 
Concept 

Demonstration and 
Validation 



Constraints exist which affect how the Strategic Defense System can be 
demonstrated. Although the 1972 Anti-Ballistic Missile Treaty permits 
research, it limits the development, testing, and deployment of different 
types of strategic defense systems and components. According to sdio's 
1989 report to the Congress on SDI compliance with the treaty, research 
includes conceptual design and laboratory testing. Development occurs 
after research but precedes full-scale testing of systems and components 
designed for actual deployment The development, testing, and deploy- 
ment of launchers, interceptors, and radars are restricted. Also, the 
extent to which the system can demonstrate its ability to interact in a 
nuclear environment is constrained by limits on nuclear testing. 

Because the Strategic Defense System cannot be tested in its operational 
environment, many system capabilities must be demonstrated through 
computer modeling and simulation. For example, in simulations, 
software models would mimic the behavior of sensors and weapons and 
be used in place of the actual elements to evaluate system performance. 
A ballistic missile attack from launch to impact must also be simulated 
in software to prompt the element models to respond. Eventually, some 
of these tests would involve actual prototypes 5 of weapons and sensor 
hardware and software interacting in the simulation. This type of inte- 
grated system-level testing would be used to evaluate the performance 
of elements within th** context of the entire Strategic Defense System. 
Thus, the ability to simulate the interaction of the systems' hundreds of 



built during the concept 
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computers with their many millions of lines of software code becomes of 
paramount importance in demonstrating whether the system can per- 
form its mission. 

Early in the Sdi program, SDfO recognized the need for integrated system- 
level testing prior to a full-scale development decision. Defense policies 
state that models or simulations can support a full-scale development 
decision and that prototyping is one of the most powerful tools available 
for detmtuning system feasibility and capability. Consequently, sdio ird- 
taally planned to build a simulation framework to conduct tests using 
software and hardware prototypes, sdio studies and contractor docu- 
ments have supported the need for such test capabilities before a full- 
scale development decision. However, because sdio believed building 
such a simulation framework would be very complex, it explored other 
approaches for demonstrating and validating the Phase I system. 



sdio decided instead to use multiple test beds. 6 Under this approach, 
each test bed would address critical strategic defense issues such as 
system performance, command and control, communications, and 
sensing and tracking. The system performance test bed, known as the 
System Simulator, would be at the core of this test environment The 
System Simulator, using computer models of individual elements, would 
perform end-to-end, system-level tests in which all aspects of Phase Fs 
performance would be evaluated while under a simulated ballistic mis- 
sile attack. Although the System Simulator, or some other simulation 
framework, might evolve to include real-time integrated hardware-, 
software-, and human-in-the-loop capabilities, SDIO has no plans to 
accomplish this prior to either the presidential decision on deployment 
or the full-scale development decision. 



Objectives, Scope, and 
Methodology 



On June 5, 1989, the Chairman, Legislation and National Security Sub- 
committee, House Committee on Government Operations, requested that 
we assess the Strategic Defense Initiative Organization's program activi- 
ties, test and evaluation requirements, and decisions on the development 
and deployment of Phase I of the Strategic Defense System. Specifically, 
the objectives of our review were to determine (1) what test and evalua- 
tion requirements were established for making development and deploy- 
ment decisions for the first phase of the Strategic Defense System and 



r Test beds are sites, facilities 



ie National Test Bed. 
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(2) whether the current program approach will meet those requirements 
and support a decision on whether to deploy. 

To address our objectives, we interviewed a wide range of officials 
involved in the Strategic Defense Initiative, sdio representatives 
included: (1) the Phase I System Engineer, as well as other Engineering 
Office officials, (2) test and evaluation officials, (3) Brilliant Pebbles 
Program Office officials, and (4) National Test Bed Program Office offi- 
cials. We discussed technical issues with members of the Phase One 
Engineering Team, which acts as a consultant to SDlO. Team members are 
experts in ballistic missile defense and include physicists, software and 
electrical engineers, and simulation analysts. We also met with officials 
from Martin Marietta, the prime contractor for the National Test Bed, 
and officials from General Electric Company, the system engineer and 
integration contractor for the Strategic Defense System. General Electric 
has the responsibility for defining Phase I demonstration and validation 
requirements," including those for test and evaluation. We met with 
Army and Air Force officials involved in major sdi research and devel- 
opment as well as Defense Acquisition Board officials. Additionally, we 
consulted with ballistic missile defense experts at Sandia and Los 
Alamos National Laboratories. 

Our approach was to obtain a thorough understanding of the underlying 
engineering concepts for the Strategic Defense System by (1) obtaining 
and analyzing key system design and requirements documents, (2) inter- 
viewing and consulting with experts from the ballistic missile defense 
community, and (3) visiting and assessing research and development 
sites. We reviewed pertinent management, technical, and contract docu- 
ments provided by sdio, SMO contractors, the Air Force, the Army, Los 
Alamos National Laboratory, and the Defense Science Board. We ana- 
lyzed and compared system documents such as the 1988 System 
Description Document, the 1990 System Description Document, the Bril- 
liant Pebbles System Description Document, and the Space-Based Archi- 
tecture Study. Furthermore, we analyzed and compared test and 
evaluation documents including the 1987 and 1989 Test and Evaluation 
Master Plan, the Phase I Integrated Test'PIan, the System Simulator 
requirements document, the technical specifications and software for 
the current System Simulator, the Brilliant Pebbles Integrated Test Plan, 
and the Brilliant Pebbles Test and Evaluation Master Plan, Through our 
analyses we were able to identify the chronology of events that led to 
sdio's current test and evaluation approach in support of decisions on 
deployment and full-scale development We did not, however, validate 
test and evaluation results. 
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Our work was conducted at sdio Headquarters, Washington, D.C.; the 
National Test Bed Joint Program Of fice, Falcon Air Force Base, Colorado 
Springs, Colorado; Electronic Systems Division, Boston, Massachusetts; 
the Army Strategic Defense Command, Huntsville, Alabama; Sandia 
National Laboratory, Albuquerque, New Mexico; Los Alamos National 
Laboratory, Los Alamos, New Mexico; the Phase One Engineering Office, 
Crystal City, Virginia; the Defense Acquisition Board, Washington, D.C.; 
General Electric Corporation, Blue Bell, Pennsylvania; and Martin Mari- 
etta Corporation, Colorado Springs, Colorado. 

We discussed the facts in this report with sdio officials and have incor- 
porated their comments where appropriate. However, in accordance 
with the requester's wishes, we did not obtain official agency comments 
on a draft of the report. We conducted our work from June 1989 
through May 1990, in accordance with generally accepted government 
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Unstable Architecture and Test Plans Will Not 
Support a Presidential Deployment Decision 
in 1993 



The President is scheduled to make a decision in 1993 on deploying 
Phase I of the Strategic Defense System. 1 Public Law 99-145 states that 
a Strategic Defense System may not he deployed in whole or in part 
unless the President certifies to the Congress that the system can sur- 
vive, meaning that it can maintain enough effectiveness during a war to 
fulfill its mission. However, by 1993 sdio will not have conducted inte- 
grated system-level tests designed to demonstrate that the entire Phase I 
system will work as planned, so it will not be able to support an 
informed deployment decision by the President. 

The recent inclusion of Brilliant Pebbles has fundamentally changed the 
1988 Phase I architecture by potentially reducing or eliminating the 
need for two sensors and one weapon component of the system. This 
change has significantly destabili2ed the architecture, resulting in a dra- 
matic restructuring of Phase I. According to sdio test and evaluation 
officials, the new Phase I architecture will not be solidified until 1991 
thus decreasing the level of system testing that can be performed by 
1993. Furthermore, according to sdio's Director, because of fiscal year 
1990 funding cuts, research will be scaled back, reducing the informa- 
tion available on which to make a deployment decision in 1993. 



Rrillinnt PpHKIpq ^pnrfc In January 1990, Brilliant Pebbles was formally mcorporated into the 

DLU1UUXI reULUSb OtJltua phase I design. However, exactly what role Bruuant Pebbles will play in 

Phase I and Strategic the Phase I architecture and even what pieces of Phase I will be 

DefellSG System intO dsployed is uncertain and consequently, has put the design of the Stra- 

q . . r. pn tegic Defense System into a state of flux. The result is a destabilized 
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Stable Integrated The SDI ° piiase I Svst:eni Engineer has stated that his first goal in sup- 

Architecture Needed for Portoftopi^Md^o^tdecatmhtoci^'^c^leteard 
Q , -p. , . coherent system design witii definition of all elements and utter-element 

bystem Development interfaces.'* It is important in systems development to have a stable, 

integrated architecture before detailed system requirements can be 
determined- A stable architecture provides the blueprint for design and 
development of each element. An unstable architecture causes confusion 
and increases the probability that system requirements and integrated 
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The Strategic Defense System is a "system-of-systems" and accordingly, 
the elements are all interdependent A requirement change for one ele- 
ment may affect all the others. For example, in the 1988 Phase I archi- 
tecture, the Boost Surveillance and Tracking System and the Space 
Surveillance and Tracking System had a requirement to combine their 
tracldng data to provide targeting information for the Space-Based 
Interceptor. This combined data would also be transmitted to the 
ground-based elements to help them identify targets. The interfaces 
must be precisely defined to support such data, sharing. If the Space Sur- 
veillance and Tracking System's mission changes, then all the elements 
must reflect the change as well. Accordingly, to ensure integrated 
system development, each element developer must be aware of all the 
other elements and the respective interfaces when designing each 
system- 



Brilliant Pebbles 
Destabilizes Phase I 
Architecture 



Major program restructuring is going on to incorporate Brilliant Pebbles 
and stabilize the architecture. Brilliant Pebbles is now the focal point of 
SDiO's efforts. Its effect on the Phase I architecture is far-reaching. 

As discussed in chapter 1, the 1988 Phase I architecture was made up of 
seven highly integrated elements. Command and control relied on rapid 
communicaJiGn and data sharing among elements. For example, during a 
battle, the Boost Surveillance and Tracking System would begin tracking 
boosters and would be one of several systems to alert national decision- 
makers. The Boost Surveillance and Tracking System would then 
transfer the information to the Space Surveillance and Tracking System, 
which would begin tracking and pass intercept information to the Space- 
Based Interceptor. At the same time, battle information would be sent to 
the ground-based weapons and sensors. The Ground Surveillance and 
Tracking System and the Ground-Based Radar would continue tracking 
and the Ground-Based Interceptor would destroy the warheads before 
they reenter the earth's atmosphere. Throughout the battle, command 
and control information would be centralized at the ground-based Com- 
mand Center Element 

The addition of Brilliant Pebbles fundamentally changed this highly- 
integrated approach to strategic defense. Brilliant Pebbles provides for 
several thousand self-contained interceptors that can detect and destroy 
missiles independently of the other Phase I elements, As a consequence, 
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interfaces, sensing and tracking capabilities, communications, and the 
numbers of elements needed — both in terms of types and constellations 
(numbers of satellites) — will have to be reassessed. For example, the 
Boost Surveillance and Tracking System may not be required as a Phase 
I element, and Brilliant Pebbles will add thousands of independent 
interceptors to the system. Furthermore, Brilliant Pebbles will eliminate 
the need for the Space-Based Interceptor, could reduce tracking require- 
ments placed on the Space Surveillance and Tracking System, and could 
expand the mission requirements for the Ground Surveillance and 
Tracking System. Unlike the Space-Based Interceptor, whose battle man- 
agement functions were highly dependent on ground-based command 
and control instructions, Brilliant Pebbles, after being turned on by the 
commander, could be autonomous. 



Phase I Architecture Defense officials have stated that the elements that will make up the 

Remains Uncertain Phase I architecture have not yet been solidified. However, several sce- 

narios have been given. An Office of the Secretary of Defense official 
stated that a space-based system made up of the Boost Surveillance and 
Tracking System and Brilliant Pebbles could be initially deployed as the 
Phase I architecture, followed by the Phase I ground-based elements. 
Another scenario is that the Boost Surveillance and Tracking System 
and Brilliant Pebbles would be deployed with a terminal interceptor 3 
(the High Endoatmospheric interceptor), which is not currently included 
in the Phase I design. 

In April 1990, the Director of sdio told the Congress that because of Bril- 
liant Pebbles potential for independent operation, the Phase I architec- 
ture may not require the Boost Surveillance and Tracking System and 
the Space Surveillance and Tracking System or may require less capa- 
bility from them. However, in May 1990, sdio officials stated that the 
Phase I architecture will include Brilliant Pebbles and all the 1988 Phase 
I architecture elements, with the exception of the Space-Based Inter- 
ceptor. The differences among such scenarios and the uncertain archi- 
tecture point to the instability of basic concepts underlying the Strategic 
Defense System. 



Unless the architecture is stabilized and the respective elements are 
developed, designed, and tested as an integrated system, sdio will not 
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know whether the system will work as planned. The process of intercon- 
necting the Phase I elements so that together they can effectively 
accomplish the strategic defense mission is called system integration. 
Successful system integration should result in the design and delivery of 
complete elements that will work in concert with the other Phase I ele- 
ments. Two of the elements, however, are preceding the rest of the 
system. Because of presidential guidance, sdio has expedited the 
research and development of Brilliant Pebbles. Consequently, Brilliant 
Pebbles has been placed in an accelerated acquisition program. The 
Boost Surveillance and Tracking System, which may or may not be 
needed if Brilliant Pebbles is successful, is approaching a full-scale 
development decision. Defense has requested S265 million for full-scale 
development of the Boost Surveillance and Tracking System in fiscal 
year 1991. The Boost Surveillance and Tracking System is a multi-mis- 
sioned satellite that supports other Defense programs. Defense docu- 
ments show that sdio wUl provide $1.3 billion (or 99 percent) of the 
system's funding through fiscal year 1991. The Boost Surveillance and 
Tracking System's specific functions for the Strategic Defense System 
are to detect missile launches, acquire and track boosters, and assess the 
number of boosters and warheads that have been destroyed. If Brilliant 
Pebbles and the Boost Surveillance and Tracking System are allowed to 
advance before the requirements of the other elements are clearly 
defined, significant integration problems and costs could emerge. 



Phase I System Test Plans SDro test ^ evaluaE "on officials claim that including Brilliant Pebbles in 
smrf ftpcnlte Mav "Rp Mnnf Phase 1 has destabilized the architecture and that a firm design will not 
<mu xua»iu u> uuiy dc muuL ^ availab]e beforfi 1991 For example ^ y nave ^^ ttat a p^^. 

nary Systems Requirements Review that was scheduled for May 1990 
will be deferred for at least a year. Furthermore, sdio test and evalua- 
tion officials are rewriting test plans to incorporate Brilliant Pebbles. 
Because most of the test and evaluation efforts have been based on the 
1988 Phase I architecture, much of the data and analysis may no longer 
be relevant, For example, if, as mentioned above, the Space Surveillance 
and Tracking System's mission is changed and it no longer provides 
tracking information, then all of the modeling and simulation done so far 
is useless because all the interfaces and data paths have changed. Any 
change to one element causes this rippling effect across the system. 

Further, system-level tests and the resulting requirements have not 
included Brilliant Pebbles. Accordingly, system tests must be rewritten 
- - and actual simulations rerun to include them. Because the other Phase I 

elements have not included Brilliant Pebbles in their tests, analyses, and 
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development, element test plans and simulations must also be 



SDTO Will Not Perform SDI& " funde ^ stu dies and test plans cite the importance of conducting 
ouiu m inui reiiuim jn^grate^ system-level tests in real time, usix^g actual system hardware 

IntCgraied oySteltl- and software with human intervention. For example, the Advanced Sim- 



1993 



to see how they work in real-time operations. Moreover, SDiO's Inte- 
grated Test Plan notes the importance of such tests before making a 
decision on full-scale development According to the plan, integrated 
system testing minimizes costs by integrating and building upon existing 
experiment assets, data, and results, and sharing a common test bed- 
However, an integrated system-level test in real time using hardware 
and software and a human-in-the-loop will not be demonstrated prior to 
the planned 1993 decision on deployment SDio officials feel they will be 
able to support a presidential deployment decision with less information 
than was originally desired, but state that this will be done at increased 
risk. 



Value of Integrated 
System-Level Testing 



Integrated system-level testing will help confirm whether the individual 
elements and the strategic defense commander can successfully interact 
in real time. Actual hardware and software in the test environment will 
provide significantly more information about system performance than 
a model. Such testing would put system components in a realistic system 
environment for detecting problems early during design and develop- 
ment, sdio officials decided that such testing will be performed in the 
System Simulator (or some other simulation framework) and will pro- 
vide the basis for system design studies and end-to-end validation of 
overall system performance. Simply put, the System Simulator will 
increase sdio's confidence that when the independently designed and 
developed elements become operational, they will integrate effectively 
into a Strategic Defense System. 

In 1 988 sdio established the Stellar Task Force to address the question 
. of how best to evolve to such an integrated system test environment. 
While the task force fully supported the use of the System Simulator for 
system-level tests, it concluded that trying to start off using only one 
test environment was too complex an undertaking. It recommended 
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breaking the problem into pieces and developing several test beds.* By 
using several test beds, the system could be broken into units, each 
focusing on a different part of the system. For example, a communica- 
tions test bed would enable sdio to conduct a series of tests on all of the 
elements* abilities to effectively communicate. 

The task force's rationale for this approach was that detailed test 
results from each test bed could be entered into the System Simulator's 
data bases for more accurate system-level testing. The elements will be 
represented at the System Simulator by software mode's rather than 
actual element hardware and software. SDio does plan to use several test 
beds and then evolve to integrated system-level testing at the System 
Simulator, incorporating actual element hardware and software in a 
real-time environment However, integrated testing will not occur before 
the President's scheduled 1993 decision. 



System-Level Test Results 
Based on Immature 
Element Models 



A stable architecture with defined system and element requirements is 
required before software models can be developed to validate each of 
the elements and the system as a whole- However, even prior to the 
inclusion of Brilliant Pebbles, the sdio Phase I System Engineer had 
stated that detailed system requirements for all Phase I elements will 
probably not be set until 1 994 or 1995. Therefore, element software 
models cannot sufficiently demonstrate how the various elements will . 
work until detailed requirements are defined. As things stand, test 
results available in 1993 will be based on limited software representa- 
tions of the elements. For example, Air Force officials have stated that 
they have not been provided detailed ballistic missile defense require- 
ments for the Boost Surveillance and Tracking System. Without such 
requirements, system developers cannot develop an accurate software 
representation for system-level testing. Consequently, it is unclear 
whether detailed requirements will be determined, a model developed, 
system testing completed, and results analyzed in sufficient time to sup- 
port the 1993 deployment decision. Therefore, not only will system-level 
tests not be conducted in real time or be integrated with pieces of hard- 
ware and software, but the tests will be based on simulations involving 
immature, possibly inaccurate element models. 
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SDIO Asserts Funding 
Cuts Will Reduce 
Information Needed 
for an Informed 
Decision in 1993 



According to sdio, an informed executive decision on deployment is con- 
tingent, in part, on minimum funding levels. At one time, this decision 
was to occur in the same time frame as Defense's formal, full-scale 
development decision. In most major acquisitions, a deployment decision 
is made after full-scale development is completed and the system has 
moved into the next stage. (See app. I.) According to sdio's Director, an 
informed decision on deployment is contingent on funding levels of S4.6 
billion for fiscal year 1990 and $33 billion over fiscal years 1990-1994. 
He stated that any funding reduction would reduce confidence in the 
deployment decision and would have increasingly serious consequences 
for the sdi program. For example, the Director stated that a 20 percent 
funding cut would reduce the confidence in making a decision on deploy- 
ment due to cutbacks in research. He further stated that emerging con- 
cepts, especially Brilliant Pebbles, would not be fully explored. Hence, 
the space-based architecture would not be completely defined, and ini- 
tial system development and deployment schedules would be delayed at 
least 2 years. 

The Congress reduced SDiO's fiscal year 1990 budget request by 20 per- 
cent (from S4.6 billion to S3.6 billion) because of congressional concern 
for overall fiscal constraints and sdio's major uncertainties for fiscal 
year 1990. Consequently, sdio initiated a major replanning and restruc- 
turing strategy to identify program priorities and impacts, and sdio 
delayed its full-scale development decision for 2 years. Many contracts 
were reduced, delayed, or cancelled, resulting in scaled-back demonstra- 
tion and validation activities that, according to sdio, will provide less 
information than was originally envisioned to support an informed 1993 
deployment decision. For example, the identification of system-level 
demonstration and validation requirements in support of system-level 
tests will not be established until fiscal year 1992; and by 1993, testing 
on command and control functions will not be as thorough as originally 
planned. Further, because the new Phase I architecture will not be 
defined until 1991, the elements will be less developed and their designs 
less detailed, therefore, less able to support system-level testing. Finally, 
the system's communications network will not be tested for real-time 
operations. Nevertheless, sdio officials have stated that they will be able 
to support a presidential deployment decision but with increased risk. 
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Program Management Oversight Has 
Been Reduced 



SDi program decisions have been made without the formal review and 
approval of the Defense Acquisition Board (dab), increasing the risk that 
program objectives will not be achieved. Defense, recognizing the com- 
plexity, uniqueness, and cost of the Strategic Defense System, required 
additional dab oversight and directed additional monitoring of sdio's 
operational test and evaluation activities. Despite the more stringent 
oversight requirements, the 1989 annual dab review did not occur as 
planned. Consequently, fundamental changes to the Phase I architecture 
and reductions in test and evaluation requirements have occurred 
without formal dab review and oversight. 



Oversight for Phase I 
More Stringent Than 
Typical Acquisition 



As discussed in chapter 1, the dab oversees major Defense acquisition 
programs. As part of its oversight responsibility, the dab conducts 
formal re nsv/s at major program milestones to ensure that programs 
are ready to proceed into more advanced stages of development and 
that proposed program plans for later stages follow sound acquisition 
management practices. Among the issues typically considered in a dab 
mi'estone review are cost, requirements growth, schedule delays, threat 
assessment, acquisition strategy, and test and evaluation plans and 
results. As part of the Defense acquisition process, dab reviews are to 
help reduce the risks associated with major systems acquisitions. 

Phase I, however, is not a typical acquisition. The complexity, cost, and 
uniqueness of the Strategic Defense System has prompted the need for a 
more stringent acquisition process. For example, in 1987 the dab devel- 
oped a guidance document called the Implementor which provides a 
framework for additional oversight Although formal acquisition 
reviews by the dab are usually oily required at major milestones, the 
Implementor expanded the dab's oversight responsibility to include 
yearly reviews, and lists additional documentation soro must submit to 
the dab to ensure program goals are being met 

In addition to the increased Dab oversight, Defense also decided in 1987 
to establish an oversight group under its Director for Operational Test 
and Evaluation to monitor sdio's test and evaluation activities. Each 
stage of the acquisition process, as well as each milestone decision, is to 
be supported by test and evaluation. The purpose of test and evaluation 
is to help ensure the timely development, production, and fielding of 
systems that meet the users' requirements and perform as intended. The 
dab relies on information from test and evaluation activities to deter- 
mine whether a system is ready to enter the next stage of development. 



1989 Annual DAB 
Review Did Not Occur; 
Decisions Have Been 
Made Without DAB 
Review and Approval 



The Implementor the ms imposed on the sor program states that the 
Phase I program baseline must be reviewed annually by the dab. These 
annual reviews were to include cost, schedule, technical performance, 
and operational assessments. The annual dab review scheduled for 
October 1989 was delayed until December and later cancelled. According 
to Defense officials, the review was cancelled because the Fiscal year 
1990 budget had not yet been finalized, and because Defense was reas- 
sessing the amount of oversight needed for the Strategic Defense System 
acquisition. The review has been rescheduled for June 15, 1990. Fur- 
ther, significant issues have not been formally reviewed by the dab as to 
their impact on the SDI program. These issues include the October 1989 
Space-Based Architecture Study's conclusions and recommendations and 
the dramatic effect of incorporating Brilliant Pebbles into Phase I. 

The Space-Based Architecture Study was initiated to review the feasi- 
bility and consequences of incorporating Brilliant Pebbles into Phase I 
with special emphasis on the space-based elements — the Space-Based 
Interceptor, the Boost Surveillance and Tracking System, and the Space 
Surveillance and Tracking System. The study concluded that Brilliant 
Pebbles was feasible, but recommended that research continue on both 
Brilliant Pebbles and the Space-Based Interceptor- Additionally, the 
study concluded that if Brilliant Pebbles were incorporated, major 
changes should be made to system requirements that could significantly 
affect program cost and schedule. For example, the study concluded 
that (1) the Boost Surveillance and Tracking System's ballistic missile 
defense requirements could be reduced; (2) the Space Surveillance and 
Tracking System is an essential asset for Phase I but may need to be 
modified; and (3) the Ground Surveillance and Tracking System's capa- 
bility will have to be increased. While sdio officials have considered the 
study's results, the dab has not formally reviewed the study and deter- 
mined its consequences on the program's design, cost, and schedule. 

Under Defense's current approach, major changes to Phase I have 
occurred without high-level Defense oversight and management review. 
Although the study recommended that research continue on both space- 
based weapons, sdio decided to include Brilliant Pebbles in the Phase I 
architecture. This fundamental change to the program baseline was 
made without formal dab review and oversight. As noted in chapter 1, 
this addition will have far-reaching impacts that will ripple across the 
Strategic Defense System; system interfaces, sensing capabilities, com- 
munications, and element requirements will all change significantly. 
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Program Management Oversight H- 



SDIO has also reduced the testing requirements for both the System Simu- 
lator and the Pilot Command Center without dab review, sdio initially 
established stringent System Simulator requirements that would demon- 
strate an end-to-end, real-time system test with hardware-, software-, 
and human-in-the loop in support of a full-scale development decision. 
However, the System Simulator is now only required to demonstrate an 
end-to-end simulation using software representations of the system ele- 
ments instead of actual pieces of hardware and actual system software 
operating in real time. Further, the Pilot Command Center, sdi's test bed 
to demonstrate battle management and command, control, and commu- 
nications functions in real time, was to include actual sensor and 
weapon hardware-in-the-loop testing. Current test plans do not include 
actual sensor and weapon interfaces. Since these significant reductions 
in demonstration and validation requirements have not been subject to 
dab review, high-level Defense Department officials have not passed 
judgment on whether this proposed approach will provide sufficient 
information to make an informed deployment decision. 



Congress Cuts 
Funding for 
Operational Testing 
and Evaluation Until 
Architecture Is 
Solidified 



The Congress has shown long-standing interest in how major weapon 
systems perform and the adequacy and timeliness of operational test 
and evaluation- As early as 1971. the Congress enacted legislation 
requiring Defense to provide the Congress with data on operational test 
and evaluation results of major weapon systems before committing 
major production dollars. 

In 1983, Congress enacted additional legislation creating the Office of 
the Director of Operational Test and Evaluation. Defense directed this 
office to preside an independent assessment of system-level testing for 
the Strategic Defense System. This office is to provide independent over- 
sight, coordinate the military services' planning and execution of opera- 
tional tests, and objectively report on test results to Defense and 



The Congress expects independent oversight and objective reporting of 
operational test results before it is willing to commit to production deci- 
sions. However, the Congress, in November 1989, expressed concern 
about the constantly changing design of the Strategic Defense System 
and, given these changes, questioned what value the Office of Opera- 
tional Test and Evaluation could provide to the Strategic Defense 
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t that time. According to the 1990 Authorization Act 



..-the need for a Strategic Defense Initiative operational test and evaluation activity 
is premature in view of the shifting SDI architectures, plans and priorities. Accord- 
ingly, the conferees agree to reduce the operational test and evaluation authoriza- 
tion by S 12.415 million from the requested amount and direct the Department of 
Defense not to obligate any appropriated funds for such an activity until it has been 
formally authorized in law. 

We agree that such testing is premature given the fluid and constantly 
changing nature of the Strategic Defense System architecture. However, 
once the architecture is set, early system-level operational test and eval- 
uation will help ensure that the acquisition process, which includes 
system-level test plans and methodologies, is not only monitored but 
also assessed. Furthermore, such oversight will help ensure that com- 
plex system-level test results, justifying a full-scale development deci- 
sion, will be understood. 



j Authorizing Appropriations for Fiscal Year 1980 for Military Activities of the Department of 
Defense, House of Representative Report No. itK-331. 101st Congress, 1st Session (Conference 
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Conclusions and Recommendations 



The Strategic Defense Initiative Organization is engaged in one of the 
most complex and technologically challenging efforts ever undertaken. 
Tens of billions of dollars will be needed to develop and deploy the first 
phase of a Strategic Defense System. The first phase is designed to pro- 
tect the United States from only a portion of a massive Soviet nuclear 
ballistic missile attack. 

The Phase I architecture of the Strategic Defense System remains in a 
state of flux, primarily because Brilliant Pebbles, a new space-based 
weapon, was added to the system. The importance of a stable architec- 
ture cannot be overstated. An architecture that is not well defined runs 
the risk that system requirements will net be met and that the system 
will not perform as intended. Unless the individual Strategic Defense 
System elements are designed, developed, and tested as an integrated 
system, sdio will not know whether the system will work as intended. 

Brilliant Pebbles has greatly changed the Phase I architecture and high- 
lights the instability of sdio's design for the Strategic Defense System. 
Instead of the highly integrated, interdependent architecture envisioned 
for Phase I, Brilliant Pebbles is to provide more autonomy and require 
less interdependence among elements. As a result, element interfaces, 
surveillance and tracking capabilities, communications, and the numbers 
of elements will change. According to the Phase I System Engineer, even 
before the sddition of Brilliant Pebbles to the architecture, detailed 
system requirements for all Phase 1 elements would probably not be 
completely defined until 1994 or 1995. Nevertheless, Defense has 
requested $265 million for full-scale development of the Boost Surveil- 
lance and Tracking System in fiscal year 1991. While the system is being 
developed to meet other Defense missions, we believe that a decision to 
enter full-scale development for the Boost Surveillance and Tracking 
System before the Phase I architecture had been solidified and its bal- 
listic missile defense requirements defined would be premature. Such 
decisions in other Defense programs have led to significant cost 
increases, schedule delays, and performance shortfalls. 

Phase I system-level test and evaluation activities will be significantly 
affected as well. System-level testing and evaluation are critical to 
making decisions on the continued development and deployment of a 
Strategic Defense System. However, the majority of system test and 
evaluation efforts so far have been based on an earlier Phase I architec- 
ture that did not include Brilliant Pebbles. Consequently, much of the 
current test data and analyses will not be applicable to the new architec- 
ture, and many of the tests will likely have to be redesigned and 
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repeated. Even though initial sdio test plans required that real-time inte- 
grated system-level tests be performed prior to a full-scale development 
decision and a presidential deployment decision, sdio officials now claim 
such tests will not be conducted prior to either decision. We believe 
sdio's initial plans could significantly reduce the risks involved in 
making critical development and deployment decisions for such a costly 
and technically complex system. 

The President is scheduled to make a decision in 1993 on deploying 
Phase I of the Strategic Defense System. The President will have a range 
of options including deploying, delaying, or cancelling the Phase I 
system. Public Law 99-145 states that the system may not be deployed 
unless the President determines that the system is sufficiently capable 
of fulfilling its mission. Given that the architecture continues to change, 
the program is falling behind schedule, and the amount of test and eval- 
uation data has been reduced) we do not believe that SDIO will be able to 
give the President enough information to support a 1993 decision to 
deploy Phase I. 

We are not alone in raising concerns about the Strategic Defense 
System's changing architecture. The Congress has been troubled by the 
constantly changing design of the Phase I system and has questioned the 
value of operational test and evaluation oversight at this time. The Con- 
gress was so concerned that for fiscal year 1990 it not only reduced sdi 
program funding by 20 percent (from $4.6 billion to $3,6 billion), but 
also cut funding for the Office of Operational Testing and Evaluation, 
citing such activities as premature in view of the shifting Strategic 
Defense System architecture, plans, and priorities. 

The federal government has repeatedly found that major acquisitions 
such as the Strategic Defense System are extremely difficult and com- 
plex undertakings, fraught with high risks. Indeed, many have 
floundered. A principal element needed to successfully design and 
develop highly integrated systems is the commitment and oversight of 
agency leaders. To provide that oversight they must have the informa- 
tion they need— such as program cost, requirements growth, schedule 
delays, acquisition strategy, and test and evaluation plans and results— 
to effectively guide the acquisition effort 

High-level oversight of the sdi program, however, is not occurring. 
The Defense Acquisition Board, which oversees major Defense acquisi- 
tion programs, has not reviewed major changes to the Phase I architec- 
ture and to the system's test and evaluation requirements. The dab's 
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1989 annual program review was cancelled. According to Defense offi- 
cials, the review was cancelled because the fiscal year 1990 budget had 
not yet been finalized, and because Defense was reassessing the amount 
of oversight needed for the Strategic Defense System acquisition. We do 
not believe these reasons justify cancelling scheduled oversight reviews 
considering the dramatic, fundamental changes in the Phase I design 
and the significant reductions in system-level test and evaluation 
requirements. As a result, significant changes, such as the addition of 
Brilliant Pebbles to the Phase I architecture and the results of the 
October 1989 Space-Based Architecture Study, have not been subject to 
formal review. 



Kecommendations to 
the Secretary of 
Defense 



We recommend that the Secretary of Defense advise the President to 
defer a decision to deploy any element of the Phase I system until SDro 
has stabilized the architecture and has demonstrated the effectiveness 
of the system through integrated system-level tests in real time, using 
system hardware and software prototypes with human intervention. 
Additionally, the Secretary should ensure that required oversight by the 
Defense Acquisition Board be followed. The Board should more closely 
monitor system design, development, testing, and evaluation. Finally, 
when the program's architecture is stable, the Secretary should request 
that the Congress reinstate funding for the Director of Operational Test 
and Evaluation to provide an independent system-level assessment for 
the Strategic Defense System. 



Recoimnendation to 
the Congress 



We recommend that the Congress not fund full-scale development for 
any element of Phase I until sdio has stabilized the architecture and has 
demonstrated the effectiveness of the system through integrated 
system-level tests in real time, using hardware and software prototypes 
with human intervention. This would include not providing S265 million 
for the Boost Surveillance and Tracking System in fiscal year 1991. 
However, if Defense needs the Boost Surveillance and Tracking System 
for another mission, independent and separate from the Strategic 
Defense System, it should be justified and funded to meet that mission 
and should no longer be considered an element of the Strategic Defense 
System. 
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Department of Defense Major Systems 
Acquisition Stages 



Mission area analysis and program initiation generally precede the five 
Department of Defense acquisition stages. Defense components continu- 
ally analyze their assigned mission areas to identify deficiencies (needs) 
and determine if new systems or major upgrades to existing systems are 
necessary. These analyses often result in recommendations to initiate 
new acquisition programs through the validation of a need to correct the 
deficiency. Once a need has been identified and validated and Defense 
initiates an acquisition program, the program enters the concept formu- 
lation stage. 



Concept Formulation 
Stage 



In this stage, potential requirements and alternative approaches to sat- 
isfy the need are identified and evaluated. Various types of analyses 
considering trade-offs among performance, life-cycle cost, and schedule 
are conducted to select among possible concepts to satisfy the need- 
Once a. concept has been selected, it is presented to Defense for 



Demonstration and 
Validation Stage 



In this stage, feasibility and desirability of the selected requirements 
and the system concept are further analyzed, generally using techniques 
like computer simulation, hardware prototyping, development test and 
evaluation, operational test and evaluation, or a combination of test 
methods. When the feasibility of the concept has been convincingly 
demonstrated and validated, the program enters the full-scale engi- 
neering and development stage. 



Full-Scale 
Development Stage 



In this stage, the system, including every item necessary for its logistic 
and operational support, is designed, fabricated, and tested. At the con- 
clusion of this stage, the system is ready to be produced. 



Full Pat* PrnHnPtiATi Duringthis stage the proposed system is built and released to the u 
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and Initial Deployment 

Stage 



Operations Support 
Stage 



This stage immediately follows deployment and extends until the 
system is removed from Defense inventory. Two major Defense revi 
are conducted in this stage. The first takes place 1 to 2 years after 



U Stable Design and Ade^iUte Testing 



Appendix 
Depattmei 
Acquisition Stage 



deployment to determine if operational readiness and support objectives 
are being achieved and maintained. The second, occurring 5 to 10 years 
after deployment, evaluates system capabilities and assesses whether 
major upgrades are needed or if the system should be replaced. 
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Strategic Defense System Elements of Phase I 



System element 


General 

function 


Specific functions 


Boost Surveillance 
and Tracking 
System 


Sensor 


Detect missile launches; acquire and track 
boosters; assess kills 


and Tracking 
System 


Sensor 


Acquire and track warheads and satellites; 


Ground 

Surveillance and 
Tracking System 


Sensor 


Track warheads and decoys; discriminate 
warheads from decoys; assess kills 


Ground-Sased 

R«ter 


Sensor 


Acquire and track warheads and decoys; 


Space-Based 
Interceptor 


Vfeapon 


Destroy boosters and warheads 


!;'.."■' tie-.'-.-"* 

Interceptor 


Weapon 


Destroy warheads 


Command Center 




Human decision-making; communications 
and guidance for defense system 
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A ppendix III 

Test Beds Supporting Demonstration and 
Validation of the Strategic Defense System 



The Surveillance and Tracking Test Bed will evaluate tracking and dis- 
crimination function performance for accuracy, computational require- 
ments, speed, robustness. It will also validate surveillance algorithms 



The Commu n icati ons Network Tes t Bed will validate communications 
network operations and management and validate security of the Stra- 
tegic Defense System's communicadons design. 

The Pilot Command Center will validate command structure, hardware- 
in-the-loop, and demonstrate proof of concept for mobile and fixed phys- 
ical configurations with selected sof fcware performance. 

The Gaming Test Bed will evaluate proposed command and control oper- 
ational concepts, build operator and user confidence in hardware-in-the- 
loop decision tools to support the development of the Pilot Command 



T h e System Simulator will provide the basic vehicle for system design 
studies and end-to-end validation of overall system performance. It will 
also validate software and integration capabilities. 
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